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Abstract 

We generalize a result from Volkov (2001, [23]) and prove that, on a large 
class of locally finite connected graphs of bounded degree (G, ~) and symmet- 
ric reinforcement matrices a = {o-i,j)ij^v{G)^ vertex-reinforced random walk 
(VRRW) eventually localizes with positive probability on subsets which consist 
of a complete d-partite subgraph with possible loops plus its outer boundary. 

We first show that, in general, any stable equilibrium of a linear symmetric 
replicator dynamics with positive payoffs on a graph G satisfies the property that 
its support is a complete d-partite subgraph of G with possible loops, for some 
d ^ 1. This result is used here for the study of VRRWs, but also applies to other 
contexts such as evolutionary models in population genetics and game theory. 

Next we generalize the result of Pemantle (1992, [14]) and Benai'm (1997, [2]) 
relating the asymptotic behaviour of the VRRW to replicator dynamics. This 
enables us to conclude that, given any neighbourhood of a strictly stable equi- 
librium with support S, the following event occurs with positive probability: the 
walk localizes on S" U dS (where dS is the outer boundary of S) and the density 
of occupation of the VRRW converges, with polynomial rate, to a strictly stable 
equilibrium in this neighbourhood. 



1 General introduction 

Let {Q,J^,P) be a probability space. Let (G, ~) be a locally finite connected 
symmetric graph, and let V{G) be its vertex set which we sometimes also denote 
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by G for simplicity. Let a := {ai,j)i,j(^v(G) be a symmetric (i.e a^j = aj^i) matrix 
with nonnegative entries such that, for all i, j £ V{G) 

i ^ j <^ aij > 0. 

Let {Xn)n<^N be a process taking values in V{G). Let F = {J^n)n<^N denote the 
filtration generated by the process, i.e J^n = criXo, ■ ■ ■ , Xn) for all n € N. 

For any i £ V{G), let be the number of times that the process visits 

site i up through time n G N U {oo}, i.e 

n 

Zn{i) = Zo{i) + I{x„=i}, 

m=0 

with the convention that, before initial time 0, a site i £ y{G) has already been 
visited Zo{i) G M+ \ {0} times. 

Then is called a Vertex- Reinforced Random Walk (VRRW) with 

starting point vq G V{G) and reinforcement matrix a := iO'i,j)ijeV{G) if ^0 = ^0 
and, for all n G N, 

These non-Markovian random walks were introduced in 1988 by Pemantle 
|13j during his PhD with Diaconis, in the spirit of the model of Edge-Reinforced 
Random Walks by Coppersmith and Diaconis in 1987 [1], where the weights 
accumulate on edges rather than vertices. 

Vertex-reinforced random walks were first studied in the articles of Pemantle 
(1992. |14]) and Benai'm (1997,^) exploring some features of their asymptotic 
behaviour on finite graphs and in particular relating the behaviour of the empir- 
ical occupation measure to solutions of ordinary differential equations when the 
graph is complete (i.e. when all vertices are related together), as explained below. 
On the integers Z, Pemantle and Volkov (1999, |l6j) showed that the VRRW a.s. 
visits only finitely many vertices and, with positive probability, eventually gets 
stuck on five vertices, and Tarres (2004,(18]) proved that this localization on five 
points is the almost sure behavior. 

On arbitrary graphs, Volkov (2001. |23j) proved that VRRW with reinforce- 
ment coefficients Oij = Ij^j, i, j G V{G) (again, i ^ j meaning that i and j 
are neighbours in the nonoriented graph G) localizes with positive probability 
on some specific finite subgraphs; we recall this result in Theorem [4] below, in 
a generalized version. More recently, Limic and Volkov [8] study VRRW with 
the same specific type of reinforcement on complete-like graphs (i.e. complete 
graphs ornamented by finitely many leaves at each vertex) and show that, almost 
surely, the VRRW spends positive (and equal) proportions of time on each of its 
non-leaf vertices. 

The VRRW with polynomial reinforcement (i.e. with the probability to visit 
a vertex proportional to a function W{n) = of its current number of visits) 
has recently been studied by Volkov on Z (2006,(23]). In the superlinear case 
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(i.e. p > 1), the walk a.s. visits two vertices infinitely often. In the sublinear case 
(i.e. p < 1) the walk a.s. either visits infinitely many sites infinitely often or is 
transient; it is conjectured that the latter behaviour cannot occur, and that in 
fact all integers are infinitely often visited. 

The similar Edge- Reinforced Random Walks and, more generally, self-interacting 
processes, whether in discrete or continuous time/space, have been extensively 
studied in recent years. They are sometimes used as models involving self- 
organization or learning behaviour, in physics, biology or economics. We pro- 
pose a two pages review of the subject in the introduction of [12]. For more 
detailed overviews, we refer the reader to surveys by Davis [5], Merkl and Rolles 
|10j . Pemantle [15] and Toth [19], each analyzing the subject from a different 
perspective. 

Let us first recall a few well-known observations on the study of Vertex- 
Reinforced Random Walks, and in particular the heuristics for relating its be- 
haviour to solutions of ordinary differential equations when the graph is finite 
and complete (i.e. when all vertices are related together), as done in Pemantle 
(1992,[14]) and Benaim (1997,[2]) . 

Let us introduce some preliminary notation, without any assumption on 
(G, ~) locally finite connected symmetric graph, possibly infinite. For all x = 
(xi)i6y(G) eM^(^), let 

S{x) := {i G V{G)/ Xi / 0} 
be its support. For all x G M^^*^) such that S{x) is finite, let 



JVi(rc) := 



E 



H{x) = Yl 



XiNi{x) 



and, if H{x) / 0, let 



Let 



and let 



■k{x) :- 



XiNi{x) 



e := |x G M^(^) s.t. \S{x)\ < cx)} 



(2) 



A := <^ 



X G 



pV(G) 



n 



s.t. Xi = 1 



i&V{G) 

be the nonnegative simplex restricted to elements x of finite support. 

Assume for now that G is a general finite graph, and define, for all n G N, 
the vector of density of occupation of the random walk at time n 



v{n) 



where no := ^^j^viG) ^oU) > 0, taking values in the nonnegative simplex A. 
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Let L ^ 1. For all n G N, the goal is to compare v{n + L) to v{n). If n ^ L, 
then the VRRW between these times behaves as though v{k), n ^ k ^ n + L, 
were constant, and hence approximates a Markov chain which we call M{v{n)). 

Then 7r(f(n)) G A is the invariant measure of M{v{n)), which is reversible 
(trivially H{v{n)) > since v{n)i > for all i, so that 7r(w(n)) is well-defined). 
If L is large enough then, by the ergodic theorem, the local occupation density 
between these times will be close to 7r{v{n)). This means that, 

{n + L)v{n + L) nv{n) + L7r{v{n)), (3) 

hence 

v{n + L)-vin)^ ——--Fivin)), (4) 
nn {v{n)j 

where 

F{x) = {xi[N,{x)-H{x)])i^ViG)- (5) 

Up to an adequate time change, (w(A;))fcgN should approximate solutions of 
the ordinary differential equation on A 

also known as the linear replicator equation in population genetics and game 
theory. 

However, the requirement that L be large enough so that the local occupation 
measure of the Markov Chain approximates the invariant measure 7r(f (n)), com- 
petes with the other requirement that L be small enough so that the probability 
transitions of this Markov Chain still match the ones of the VRRW, so that the 
heuristics breaks down when the relaxation time of the Markov Chain is of the 
order of n, which can happen in general on non-complete graphs and is actually 
consistent with the fact that the walk will indeed eventually localize on a small 
subset. An illustration of how such a behaviour can occur is given in the proof 
of Lemma 2.8 in Tarres |18j . The study of the a.s. asymptotic behaviour of the 
VRRW on an infinite graph is even more involved in general. 

Let us yet study the replicator differential equation ([6]) associated to the 
random walk on A for general locally finite symmetric graphs (G, ~). 

It is easy to check that H is a strict Lyapounov function for ([6]) on A, i.e. 
that it is strictly increasing on the non-constant solutions of this equation: if 
x{t) = {xi{t))i^G is the solution at time starting at x(0) := xq, then 

where, for all x G A, 

J{x) := 2 N,{x)F{x)i = 2 ^ Xi{Ni{x) - H{x)f. (7) 

i&S{x) ieS{x) 
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Note that the restriction of H to the equihbria of ([6]) takes finitely many values 
if G is finite (see [M] for instance) . 

Let us now deal with the equilibria of this differential equation: a point 
X = (xi)jgy((^) G A is called an equilibrium and only if F{x) = 0. An equilibrium 
is called feasible provided H{x) / 0. 

The reason why we only consider feasible equilibria x G A is that, for all 
n G N and i G G, Zn{i) ^ Ylj^i ^n{j) + '^Oj so that an accumulation point x of 
{v{n))neN in A would satisfy Ni{x) ^ (miuj^j ajj)xi for all i £ V{G), hence 

Vfe-e5(x), -V^^^^) \S{X)\ 

by Cauchy-Schwarz inequality. 

By a slight abuse of notation, we let DF{x) = {dFi/dxj)^ j(zv{G) denote both 
the Jacobian matrix oi F at x, and the corresponding linear operator on 0. Since 
A is invariant under the flow induced by F, the tangent space 

TA:= ivGS/ Y vi = 

is invariant under DF{x). We let DF{x)\ta denote the restriction of the operator 
DF{x) to TA. 

When x is an equilibrium, it is easily seen that DF{x) has real eigenvalues 
(see Lemma [T|. Such an equilibrium is called hyperbolic (respectively a sink) 
provided DF{x)\ta has nonzero (respectively negative) eigenvalues. It is called 
a stable equilibrium if DF{x)\ta has nonpositive eigenvalues. Note that every 
sink is stable. Furthermore, by Theorem [1] below, every stable equilibrium is 
feasible. 

We will sometimes abuse notation and identify arbitrary subsets H oi G to 
the corresponding subgraph (ff, ~). Given x £ V{G) and a subset A of V{G), 
we write x ~ ^ if there exists y £ A such that x ~ y. Given two subsets R and 
S of V{G), we let 

dR = {yeV{G)\R : yr^ R}, dsR = {ye S\R : yr^ R}; 

dR is called the outer boundary of R. 

A site i G V{G) will be called a loop if i ~ i, and we will say that a subset H 
contains a loop iff there exists a site in it which is a loop. 

We will say that x is a strictly stable equilibrium if it is stable and, furthermore, 
for all i G dS{x), Ni{x) < H{x). We let £s be the set of strictly stable equilibria 
of ([6]) in A. Note that x stable already implies Ni{x) ^ H{x) for all i G dS{x), 
by Lemma [H 

Given d ^ 1, subgraph (5*, ~) of (G, ~) will be called a complete d-partite 
graph with possible loops, if {S, ~) is a d-partite graph on which some loops have 
possibly been added. That is 

s = ViU ...uVd 

with 
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(i) V p e {1, . . . , 4, V j G Vp, ifi^j then i 9^ j. 

(ii) y p, q e {1,. . . ,d}, p q,yi e Vp, Vj G Fg, i ~ j. 
For all S Q G, let (P)^ be the following predicate: 

(P)5(a) {S, ~) is a complete d-partite graph with possible loops. 

(P)g(b) If a ~ a for some a £ S, then the partition containing a is a singleton. 

(P) g{c) U Vi, 1 ^ i ^ d are its d partitions, then for all i, j G {1, . . . , d} and 

a, a' G Vi,, (3, P' G Vj, a^^p = aa',/3'. 

Theorem 1 If x G A zs a stable equilibrium of ^B^, then x is feasible and (P)s(x) 
holds. 

In the case a = {cLij)ij^v{G) = (Ii~j)ijey(G) the following Theorem [2] pro- 
vides a necessary and sufficient condition for x G A being a stable equilibrium. 
Theorems [1] and [2] are proved in Section [2.21 

Theorem 2 Assume aij = Ij^j for all i, j G G, and let x = {xi)i^G £ ^• 

//(5'(x),~) contains no loop, then x is a stable (resp. strictly stable) equilib- 
rium if and only if there exists d ^ 2 such that 

(i) (S'(x),~) is a complete d-partite subgraph, with partitions =: V\, ... Vd, 

(ii) EiGVp = l/"^ fo"^ allpG {1,..., d}, 

(iii) Vi G dS{x), N^{x) ^ (resp. <) 1-1/d (= H{x)). 

If {S{x),^) contains a loop, then x is a stable (resp. strictly stable) equi- 
librium if and only if (S'(x),~) is a clique of loops (resp. with the additional 
assumption: \/j G dS{x), Nj{x) < 1 or, equivalently, d{j} ^ ^i^))- 

Remark 1 Jordan [6] independently shows, in the context of preferential du- 
plication graphs, that conditions (i)-(iii) in Theorem [2] are indeed sufficient for 
X G A being a stable equilibrium when loops are not allowed. 

Remark 2 A connection between the number of stable rest points in the replica- 
tor dynamics (or of patterns of evolutionary stable sets (ESS's)) and the numbers 
of cliques of its graph was made by Vickers and Cannings [HI [22], Broom [3] et 
al., and Tyrer et al [20], motivated by the study of evolutionary dynamics in 
biology. 

A consequence of Theorem [Tj is that supports of stable equilibria are generi- 
cally cliques of the graph G. More precisely assume that the coefficients (ajj)jjgG 
are distributed according to some absolutely continuous distribution w.r.t. the 
Lebesgue measure on symmetric matrices. Then the supports of stable equilibria 
are a.s. cliques of the graph G (i.e. any two different vertices are connected), as 
a consequence of (P)^^^.-) (a) and (c). 
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The following Theorem [3] states that, given any neighbourhood M{x) of a 
strictly stable equilibrium x ^ Eg then, with positive probability, the VRRW 
eventually localizes in 

T{x) := S{x)UdS{x), 

and the vector of density of occupation converges toward a point in M{x), which 
will not necessarily be x (there may exist a submanifold of stable equilibria in 
the neighbourhood of x). Note that this will imply, using Remark [21 that the 
VRRW generically localizes with positive probability on subgraphs which consist 
of a clique plus its outer boundary. 

More precisely, let us first introduce the following definitions. For all S C 
V{G), let 

S{S) :={veA s.t. S{v) = S}. 
For any open subset [/ of A containing x £ A, let C{U) be the event 

C{U) := {v{oo) := lim v{n) exists and belongs to £s riS{S{x)) n U}. 

n— >oo 

Let TZ be the asymptotic range of the VRRW, i.e. 

TZ := {i £ G s.t. Zoo{i) = oo}. 
For any random variable v taking values in A, let 

Adiv) := I^Vi G dS{v), converges to a (random) limit G (0, oo)|' . 

Theorem 3 Let x € A be a strictly stable equilibrium. Then, for any open 
subset U of A containing x, 

p({7^ = T{x)] n c{u) n Aa{v{oo))) > 0. 

Moreover, the rate of convergence is at least reciprocally polynomial, i.e. there 
exists V := Cst(x,a) such that, a.s. on C{By^{e)), 

lim (v{n) — v{oo))n'^ = 0. 

n — ^oo 

Theorem [3] is proved in Section [2.3[ It naturally leads to the following ques- 
tions. 

Firstly, are all the trapping subsets always of the form T{x) for some x G 
Egl The answer is negative in general: let us consider for instance the graph 
(Z, ~) of integers, to which we add a loop ~ at site 0, with Ojj := Ij^j. 
Then x := (I{i=o})iez is a stable equilibrium, but is not strictly stable since 
N_i{x) = Ni{x) = 1 = H(x). However Proposition below (proved in Appendix 
IA.2P shows that v{n) converges to x with positive probability, by combining 
an urn result from Athreya [1], Pemantle and Volkov [16] (Theorem 2.3) with 
martingale techniques from Tarres [18] (Section 3.1). 
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Proposition 1 Let (G, ~) be the graph of integers defined above, and aij := 
Ij^j. Then, with positive probability, the VRRW localizes on {— 2, — 1, 0, 1, 2}, 
and there exist random variables a € (0, 1), C and C > such that 



(i) 



1 



n 



n- 



■oo 



(ii) 



(Z„(-1),Z„(1)) 



(a, 1 — a), 




J n—>oo 



We conjecture that, conditionally on a localization of the VRRW on a finite 
subset, its vector of density of occupation on the subset converges to a stable 
equilibrium x oi that the asymptotic range 7^ is a subset of S{x) U dS{x) U 
d{dS{x)), and is equal to T{x) = S{x)L)dS{x) if x G Eg, which occurs generically 
on a; recall that S{x) then is a complete d-partite subgraph with possible loops 
for some d ^ 1, by Theorem [TJ A proof would require a deeper understanding 
of the dynamics of (see Lemma H]). Note that, on the integers Z 

(with standard adjacency, unlike Proposition [T|) with Oij = Ij^j, the result that 
the VRRW a.s. localizes on five sites [H] implies that stable equilibria which are 
not in £s are ruled out (otherwise six sites would be possible as well); this can 
be related to the property in this case that every neighbourhood of any stable 
equilibrium x contains a strictly stable one. 

Secondly, which subsets are of the form T{x) = S{x)UdS{x) for some x £ fg? 
We know from Theorem [1] that subsets S{x) satisfy (P)5(^.) and thus always con- 
sist of a complete d-partite subgraph with possible loops and its outer boundary 
for some d ^ 2. But (P)5(a,) is not sufficient, and the occurrence of such subsets 
also depends on the reinforcement matrix a = (ajj)ijgv{G)- Even in the case 
o = {o.i,j)i,jeV{G) = (Ii~i)ijey(G) Theorem [2] provides explicit criteria for x £ Eg, 
but the corresponding condition (iii) (when {S{x), ~) has no loops) is on x, thus 
not explicitly on the subgraph. 

We introduce in the following Definition [1] the notion of strongly trapping 
subsets, which we prove in Theorem [4] to always be such subsets T{x) for some 
X £ Eg -and actually for all x G S, where S C is defined in the statement 
of the theorem. As a consequence, by Theorem [3l the VRRW localizes on these 
subsets with positive probability. The result is thus a generalization to arbitrary 
reinforcement matrices of Theorem 1.1 by Volkov (2001, [23]) when a^j- := Ijj^jj, 
in which case the assumptions of Definition [1] obviously reduce to (c) or (c)'. 



Definition 1 A subset T C V{G) is called a strongly trapping subset of (G, ~) 
ifT = S U dS, where 




}, with common value =: as 
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either (c) (i) S is a complete d-partite subgraph of G for some 2, 
with partitions Vi, . . ., V^, 
(ii) Vj G dS, 3p G {1, . . . , d} and i ^ S\Vp such that j VpU {i}, 
or (c)' S is a clique of loops, and \/j G dS, d{j} 2 S. 

Theorem 4 Let T he a strongly trapping subset of (G, ~); then the VRRW has 
asymptotic range T with positive probability. 

More precisely, assume T = SU dS, where S satisfies conditions (a)-(c) or 
(c)' of DefinitionUl and let us use the corresponding notation. Let 

S := J X G S{S) s.t. Yj^'^ = for all I q d\ 
[ i^V, J 

ra := d/{d - 1) 

i/ (S*, ~) contains no loops, and T, := S{S), r^ := 1 otherwise. 

Then, for any G S and any neighbourhood J\f{x) of x in S, with positive 
probability there exists random variables y G M{x) and Cj > 0, j G dS, such that 

(i) VRRW eventually localizes on T , i.e. IZ = T 

(ii) Zn{i)/n — > yi for all i £ S 

(iii) Zn{j) ~ C-n'''*^-^ ''''^^'/"^ 

n— »oo 

Theorem |4] is proved in Section 12.2.31 We provide in Example [T] (illustrated 
in Figured]) a counterexample showing that Theorem [3] is stronger, even in the 
case a = (li^j)jjeG- 

Thirdly, which conditions on the graph and on the reinforcement matrix a 
do ensure the existence of at least one strictly stable equilibrium x £ £s, thus 
implying localization with positive probability on T{x)l First note that, trivially, 
this does not always occur, for instance on Z when (/>(n) := ajn^^+i} is strictly 
monotone, in which case we believe the walk to be transient. 

In the case a = (Ij^j)ijgG) Volkov [23] proposed the following result, using 
an iterative construction on subsets of the graph. 

Proposition 2 (Volkov. 123^) Assume that a = (Ij^j)j.jgG, and that (G, ~) is a 
locally finite graph without loops. Then, under either of the following conditions, 
there exists at least one strongly trapping subset: 

(A) (G, ~) does not contain triangles; 

(B) (G,~) is of bounded degree; 

(C) the size of any complete subgraph is uniformly bounded by some number K . 

PROOF: Start, for some d^ 2, with any complete d-partite subgraph (5, ~) of G 
with partitions Vi, . . ., Vd (for instance a pair of connected vertices, d = 2). Let 
X e dS, S = ViU ...UVd-. 
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1) First assume that x ^ Vp for all 1 ^ p ^ d. Then, for all 1 ^ p ^ d, let jp G 
Vp^ be such that x ~ j^; iterate the procedure with the subgraph ^i^p^d{jp}^{x} , 
which is a clique, and thus a complete {d + l)-partite subgraph. 

2) Now assume there exists p such that x ^ Vp, with d{x} ^ S \ Vp. Then 
we iterate the procedure with the complete d-partite subgraph S" U {x} with 
partitions Vi, . . ., ViU {x}, . . ., V^. 

3) Otherwise we keep the same subgraph S and try another x £ dS. 

If S has remained unchanged for all x G dS, this implies that T = S U dS is 
a strongly trapping subgraph in the sense of Definition [H so that the VRRW has 
asymptotic range T with positive probability. □ 

Using a similar technique, we can obtain the following necessary condition for 
the existence of a strongly trapping subset in the case of general reinforcement 
matrices a, when the graph does not contain triangles or loops. Let us first 
introduce some notation. Let c be the distance on E{G) defined as follows: 
for all e, e' G E{G), let c(e, e') be the minimum number of edges necessary to 
connect e to e' plus one (and if e = e'). For all e = {i,j}, let C(2, e) be the set 
of maximal complete 2-partite subgraphs S C V{G) such that i, j £ V{G) and, 
for all k, I G S with k ^ I, Uk^i = aij. 

Proposition 3 Assume the graph does not contain triangles nor loops. If, for 

some e £ E(G), 

min max aui^Oe, (9) 

then there exists at least one strongly trapping subset. 
Note that ^ holds if 

max Of,! ^ Oe- 

c{e,e')^2 

Remark 3 If, for all e G E{G), ([9]) does not hold then there exists, for all 
e G E{G), an infinite sequence of edges (en)„gNo such that eo = e, e„ ~ Cn+i 
and, for all n G N, Oe„ ^ CLsn+i and ae„ < ae„+2- However, even in this case, there 
can exist a strictly stable equilibrium x G Sg (but no strongly trapping subset). 

PROOF OF Proposition O By assumption, there exist e = {i, j} and a maximal 
complete 2-partite subgraph S C V{G) containing i and j, with partitions Vi and 
V2, and satisfying conditions (a), (b) and (c)(i) of Definition [TJ For all k G dS, 
k is adjacent to at most one of two partitions, say Vi, since otherwise G would 
contain a triangle; if k were adjacent to all vertices in Vi, then it would be in 
V2, since S is assumed maximal. Hence (c)(ii) holds as well, and 5 is a strongly 
trapping subset. □ 
When the graph contains triangles, the property outlined in Remark [3l i.e. 
the existence of an infinite sequence of edges with increasing labels when there 
is no strongly trapping subset, does not hold anymore. The maximum of the 
Lyapounov function on a complete subgraph with more than two vertices takes 
a nontrivial form, which can lead to counterintuitive behaviour. 
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Figure 1: We show in Example [T] that T := {A, B,C, D, E} does not satisfy the assumptions of 
Theorem SI but is a trapping subgraph with positive probabihty by Theorems [2] and [31 The numbers 
indicated in superscript of vertices represent the Hmit proportions of visits to these vertices if v{n) 
were to converge to the equihbrium x in the example. In this case the walk would asymptotically 
spend most of the time in the bipartite subgraph S := ViU V2, where Vi := {A,C}, V2 := {B,D}, 
evenly divided between partitions Vi and V2, and vertex E would be seldom visited, of the order of 
^/n times at time n. 

We show for instance in Example [2] a case where the reinforcement matrix a 
has a strict global maximum at a certain edge, but where however there is no 
stable equilibrium at all. We believe the walk to be transient in this example. 

Example 1 Let us show, in the case a = (Ij^j)jjgG', that Theorem [3] is stronger 
than TheoremHl Consider a graph G on six vertices A, B, C, D, E and with a 
neighbourhood relation ~ defined as follows (see Figured]): A~i?~C~L'~A, 
C ^ E ^ D and E ^ F (recall that the graph G is symmetric). Let x = 
{xa,xb,xc,xd,xe,xf) := (3/8,3/8,1/8,1/8,0,0), then S{x) = {A,B,G,D} 
and dS{x) = {E}. Also, x is an equilibrium of (P)s(x) is satisfied with 
Vi = {A,G}, V2 = {B,D}, and Ne{x) = 1/4 < H{x) = 1/2, which implies that 
x is a strictly stable equilibrium by Theorem [21 hence subsequently by Theorem 
[3l that TZ = T{x) with positive probability. 

Now let us prove by contradiction that T[x) with such x does not satisfy 
the assumptions of Theorem [H above. Indeed, if T{x) = S U dS, then S C 
{A, B,C, D} since, otherwise, F would belong to T[x). Now the condition that 
for all a G dS, 3i G {1, . . . , d} and j3 & S\Vi such that a 9^ U {/?} implies in 
particular that a vertex in dS is not connected to at least two other vertices in 
S, so that a G dS cannot he A, B, G or D which are are connected to all other 
but one vertex in {A,B,G,D}. Hence S = {A,B,G,D}, but then a := E is 
connected to both partitions of S, and does not satisfy the condition mentioned 
last sentence, bringing a contradiction. 

Example 2 Let us first study the case of a triangle (G, ~), V{G) := {0,1,2}, 
~ 1 ~ 2 ~ 0, with reinforcement coefficients a := ao,i, b := 01^2, c := 00,2 > 0. 
If a < 6 + c, then the equilibrium x = (xq, xi,X2) = (1/2, 1/2, 0) is not stable. 
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since N2{x) = (6 + c)/2 > H{x) = a/2. Hence, if we assume that 

a < b + c, b < a + c, c<a + b, (10) 

then a stable equihbrium has to belong to the interior of the simplex A. A simple 
calculation shows that there is only one such equilibrium: 



c(a + 6 — c) b{a + c — b) a{b + c — a) 



X = {xo,Xi,X2) 



where 

S:={a + b + cf - 2(a2 +b^ + c^); 

6 > 0, which can be shown by adding up inequalities (6 — a)^ ^ c^, (c — a)^ ^ 6^ 
and (c - bf ^ a^. Then H{x) = 2abc/6. 




Figure 2: On the infinite graph on the figure, with reinforcement coefficient sequences {pn)n^o strictly 
decreasing and (gn)n^o strictly increasing, we show in Example [2] that, even if po = sup„>gp„ > 
sup„>Q g„, we can choose these sequences in such a way that there is no stable equilibrium in A, and 
therefore no trapping subgraph. 



Let N := Let us now consider the following graph (G, ~) with vertices 
V{G) := {i, i,i £ N} and adjacency i ~ i + 1 , + i~i and i ~ i + 1 , for 

all i G N. 

Fix e, rj, p, q > 0, n £ (0, 1), which will be chosen later. Let, for all n G N, 

n—l n—1 
fe=0 fc=0 

Note that, for all n G N, 

Now assume that the reinforcement matrix (ofc,/)fc,/ey(G) is defined as follows. 
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depending on {pn)neN and (g„)neN, for all i £ N: 

a2i,2i±l ■■= P2i, «2m,2{i+T) •= P2{j+1), 
"■Yi,2i+1 = 0-21+1, 2(1+1) •= ^2i+l 
'^2i,2l = Q'2l, 2i+1 92 j 
«2m,2i+T •= ^'2i+l 
"'2i+T, 2(t+l) •= 121+1- 

Let x G A be a stable equilibrium of ([6]). Then, by Theorem [H (P)s(x) holds, 
so that S{x) consists of two vertices or a triangle (it cannot be made of four 
vertices, because of (P)5.^^^(c)). Assume 

p < 2q, r/qe^ <p( 1- — ) . (12) 

Then, for all i G N, 

Pi < 2qi, pi+i <qi+ qi+i, qi+i < qi+pi+i, 

so that S{x) has to be a triangle. 

Assume S{x) := |2i, 2i, 2i + 1 | for some i G N; the argument is similar in 
other cases. Then 

_H{x) __H{x) _H{x) 

"^2^+1 ~ - -p,), X2i+l-^^, 

and 

N2i{x) = qiXYi + PiX2i+i = H{x) 

and, therefore, 

H{x) 

^2i+T(^) = qi+iXYi + Pi+iX2i+i = H{x) + ^--2-[(ft.+i - qi){2qi - pi) + (p^+i - p.i)qi 

2,q^ 

H{x 

if 



ri>e-^, (13) 

using that p/ {2q — p) > Pi/{2qi — pi) for all i G N. 

Hence x is not a stable equilibrium, which leads to a contradiction. 



2 Introduction to the proofs 
2.1 Notation 

We let N := Z+, N* := N \ {0}, M.X := M+ \ {0}. 
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For all y = {yi)ieG S Mp and for any finite subset A of G, let 

VA ■■= "^Vi- 

Given r G N*, let (.,.) (resp. |.|, ||.||oo) be the scalar product (resp. the 
canonical norm, the infinity norm) on M'", defined by 

r 

(a, 6)=^aj6j, \a\ = a/ (a, a), \\a\\oo 

i=l 

if a = (ai, . . . , Ur) and b = (61, ... , br). 

Given a real r x r matrix M with real eigenvalues, we let Sp(M) denote the 
set of eigenvalues of M. When M is symmetric we let M[.] denote the quadratic 
form associated to M, defined by M[a] = {Ma, a) for all a £ W. 

Given yi, y.^, we let Diag(yi, . . . , yr) be the diagonal r x r matrix of 
diagonal terms yi, . . ., yr- 

For all u, V GM, we write u = □(f) if \u\ ^ v. Given two (random) sequences 
iun)n^k and {vn)n^k taking values in M, we write u„ = Vn if u„ — converges 
a.s, and Un ~j^_>oo w„ iff Un/vn 1, with the convention that 0/0 = 1. 

Let Cst(ai,a2, . . . ,Op) denote a positive constant depending only on ai, 02, 
... Up, and let Cst denote a universal positive constant. 

2.2 Proof of Theorems [1], H and H 

Theorems [1] and [2] are a consequence of the more general three following Lemmas 
nil] and El below. 

2.2.1 Lemmas [H [2] and [3l and proof of Theorem [1] 

By the following Lemma [U if an equilibrium x £ A is stable, then the eigenvalues 
of [ttij — 2H{x)]ij^s(^j.), which depend only on a, S{x) and H{x), are nonpositive. 
This property will subsequently imply (P)s(x)' Lemmas [2] and El 

Lemma 1 Let x = (xj)jgy((^) £ A be an equilibrium. Then 

(a) DF{x) has real eigenvalues. 

(b) The three following assertions are equivalent: 

(i) X is stable 

(ii) maxSp(i:>F(a;)) ^ 

(ill) max (Sp ([aij - 2H{x\.^s{x)) U {Ni{x) - H{x),i G 55(x)}) ^ 

(c) // X is stable, then it is feasible. 

The following Lemma [2] yields an algebraically simpler characterization of 
assertion (P)^ for S C V{G); recall that, given subsets 5 and R of V{G), dsR, 
defined in Section [H is the outer boundary of R inside S. 



:= max Oj 

l<i<r 
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Lemma 2 The statement (P)g is equivalent to 

(PYs If j, k S are such that j ^ k, then, for all i £ S, aij = ai^^ 
(so that ds{j} = ds{k} in particular). 

Lemma [3] states that {P)s(^x) holds if the eigenvalues of [ajj — 2//(x)]jjg5(j,) 
are nonpositive, with equivalence if a = (Ij^j)ijgG. 

Lemma 3 Let x = (a;i)i6y(G) £ A 6e a feasible equilibrium. Then 
maxSp {[ai^j - 2H{x\.^g^^^^ ^ ^ (P)sw 

If for some c > 0, aij = cli^j for all i, j G S{x), then the above implication is 
an equivalence. 

Lemmas [H [2] and [3] are proved respectively in Sections 13.11 13.21 and 13.31 They 
obviously imply Theorem [TJ 

2.2.2 Proof of Theorem [2] 

Suppose a = (Ij^jjij-gG, and let x £ A. 

First assume that {S{x), ~) contains no loop. If a; is a stable equilibrium, then 
(■^)s'(x) thus (i) holds by Theorem [U let 14, 1 ^ k ^ d he the partitions of 
S{x). Then d ^ 2 (otherwise II(x) = and x is not feasible, thus not stable by 
Lemma d]) and, for all 1 ^ k ^ d, j G V^, 

Vk ■■= Xi = I - Nj{x) = 1 - H{x), 

so that Vk = 1/d (since X^^ffc = 1) and H{x) = 1 — 1/d, and subsequently (ii)- 
(iii) hold by Lemma [H Conversely, assume (i)-(iii) hold; then Ni{x) = I — 1/d 
for all i £ S{x), so that H{x) = ^^i^six) ~ ^~ ^/'^ x is a feasible 

equilibrium. Now (i) implies (P)5(2;) and thus (P)'s[x) Lemma[2j Hence, using 
Lemmas [1] and El x is a stable equilibrium. 

Now assume on the contrary that (5(x),~) contains one loop i ~ i. If x is a 
stable equilibrium then Ni{x) = 1 = II(x) (x equilibrium), which implies that, 
for all j G S{x), Nj{x) = 1 so that j ~ j and, subsequently, that (^(x), ~) is a 
clique of loops by (P)^^^^(b). Conversely, if (5(x),~) is a clique of loops, then 
(^)s{x) obviously holds so that, by Lemmas [1] and [3l x is stable (since H{x) = 1, 
then Ni{x) ^ H{x) for all i € V[G)). 

2.2.3 Proof of Theorem H 

First observe that 

Indeed, the proof of Theorem [2] implies that S D S[S) n Eg and, conversely, that 
if X S S, then x is a equilibrium and, by (c)(ii), for all j G dS{x), Nj{x) < H{x) 
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(= as{l — l/d) if {S{x), ~) contains no loops, = as otherwise), using assumptions 
(a)-(b) and (c)(ii) or the second part of (c)'. Also (P)s(a;) holds by (c) or (c)', 
and therefore x is strictly stable by Lemmas [iHSl The rest of the proof follows 
from Theorem [31 

2.3 Proof of Theorem [3] 

Firstly, we provide in Lemma[4]a rigorous mathematical setting for the stochastic 
approximation of the density of occupation of the VRRW v{n) by solutions of 
the ordinary differential equation ([6]) on a finite graph G, heuristically justified 
in Section [1] (see dH)). Secondly, we make use of this technique and of an entropy 
function originally introduced in [9j to study the VRRW on the finite subgraph 
T{x) when its density of occupation is in the neighbourhood of a strictly stable 
equilibrium x, in Lemmas [SHlOl Thirdly, we focus again on a general graph G - 
possibly infinite - and prove in Proposition [H assuming again that the density of 
occupation is in the neighbourhood of an element x £ Eg, that the walk eventually 
localizes in T{x) with lower bounded probability. 

In the first step, we make use of a technique originally introduced by Metivier 
and Priouret in 1987 [H] and adapted by Benaim [2] in the context of vertex 
reinforcement when the graph is complete (Hypothesis 3.1 in ^). In Sections 
I4.1H4.31 we generalize it and show that a certain quantity z{n), depending only 
on a, v{n), Xn and n and defined in ([36]) . satisfies the recursion ([37]) : 

1 F(z(n)) 

z{n + 1) = z{n) + — ■ rTT7r"rTT + ^n+i + r-„+i, 

n + no + 1 H{v{n)) 

where E(en+i | J^n) = 0. The following Lemmadl proved in Section HT3l provides 
upper bounds on the infinity norms of Cn+ii ^n+i and z{n) — v{n), and on the 
conditional variances of {en+i)i, i £ G. 

More precisely, let us break down the set of vertices of G as V{G) = S U dS, 
where {S, ~) is finite, connected, and not a singleton unless it is a loop. Let, for 
all a G M+ \ {0}, 

Ao, := {v = {vj)j(zG G A s.t. Vj ^ a for all j £ S}. (14) 



Lemma 4 For all n ^ Cst(a) and i £ G, if v{n) £ then 
/Nil II ^ C5t(a,a, IGI) 2 I ^ ^ 



n + no (n + no)^ 

(c) ||r„,,|U ^ 'f";"''?^ (d) Un) - .(n)|U ^ '^'^^ 
(n + no)^ n + no 

Note that, if G were a complete d-partite finite graph for some 1, or more 
generally if G were without loop and, for all i, j £ G with i ~ j, {i,j}Ud{i,j} = 
G, then the constants in the inequalities of Lemma H] would not depend on a > 



16 



and, as a consequence, the stochastic approximation of z{n) by ([6]) would hold 
uniformly a.s. Indeed, for all n G N, by the pigeonhole principle there exists at 
least one edge i, j £ G, i ^ j, on which the walk has spent more than 

n/|Gp times, so that v{n)i Av{n)j ^ ^+^0 under the assumption on G, 
Lemma m with S := {i,j} would yield the claim. 

In the second step, we define an entropy function Vq{.), measuring a "distance" 
between q and an arbitrary point (as can be seen by (llSp below), originally 
introduced by Losert and Akin in 1983 in [9] in the study of the deterministic 
Fisher- Wright-Haldane population genetics model, and to our knowledge so far 
only used for the analysis of deterministic replicator dynamics. Note that it 
is not mathematically a distance however, since it does not satisfy the triangle 
inequality in general. 

In the following, until after the statement Lemma [10] - and in particular in 
Lemmas [SHlQl- we assume that x £ £s and G = T{x) = S{x)U dS{x)] this choice 
will be justified later in the proof. Note that if g G N{x) n Es-, where M{x) is an 
adequately chosen neighbourhood of x, then q G S{S{x)) since x G Es, so that 
T{q) = T{x). Set S := S{x), T := T{x), and S := S(S(x)) for simplicity. 

Lemmas [5] and [6] below will imply that, given any stable equilibrium q G 
M{x) n as a reference point, Vq{z{n)) decreases in average when z{n) is close 
enough to x. Therefore, martingale estimates will enable us to prove in Lemma 
[7] that, starting in the neighbourhood of x, v{n) remains close to x with large 
probability if n is large, and converges to one of the strictly stable equilibria in 
this neighbourhood. 

For all q = {qi)ieG G 5 and y G R^C^), let 



- Eies 1i iVi/Qi) + 2?/95 if yi > 0, Vi G 5 
oo otherwise. 



Let, for all g G 5 and r > 0, 

Bv.ir) := {y G A s.t. Vg{y) < r}, B^{q,r) := {y G A s.t. \\y - q\\^ < r}. 

Then, we will prove in Section 14.41 that, for all g G 5, there exist increasing 
continuous functions ui^q, U2,q ■ M+ — > M+ such that ni^g(O) = ti2,g(0) = and, 
for all r > 0, 

Boo{q,ui^q{r)) C Bv,{r) C B^{q,U2,q{r)). (15) 
Let, for all g, z G R^^^\ 

i£S ieds 

The following Lemma [5l also proved in Section 14.41 provides the stochastic 
approximation equation for Vq{z{n)), g G 5 n iS^; we make use of notation u = 
□ (f) <^=^ |n| ^v, introduced in Section [2.1[ 



Lemma 5 Let q G SCiSs- There exist an adapted process (Cn)neN ("n-ot depending 
on q and a), and constants ui and e (depending only on q and a) such that, if 
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n ^ ni and v{n) £ By^ie), then Vq{z{n)), Vq{z{n + 1)) < oo, E(Cn+i I -^n) = 
and 

+ 1)) = VMn)) + y^'l^ - {q, Cn+i) + 2(6„+i)a5 + □ (^^^^ 

(17) 

Lemma [6] below, proved in Section 13.41 provides estimates of the Lyapounov 
function H, and of /.(■)) the neighbourhood of a strictly stable equilibrium. 
It will not only be useful in the proof of Lemma [7] below, stating convergence of 
v{n) with large probability, but also for Lemma[8]on the rate of this convergence. 



Lemma 6 There exists a neighbourhood M{x) of x in A such that, for all q G 
M{x)n£s, yGM{x), 

(a) Cst(x, a)J{y) ^ H{q) - H{y) ^ Cst(x, a)J{y), (18) 

(b) - [H{q) - H{y) + Cst{x, a)yas\ ^ Iq{y) ^ -[H{q) - H{y) + Cst(x, a)yas] ^ 

(19) 

Remark 4 Lemma [6] implies that y G J\f{x) is an equilibrium iff H{y) = H{x). 
Also note that the maximality oi H aX x G £s is not global in general. For in- 
stance, in the counterexample at the end of Section[Tl x := (3/8, 3/8, 1/8, 1/8, 0) G 
£s but, letting y := (0,0,1/3,1/3,1/3), H{y) = 2/3 > H{x) = 1/2. 

The following Lemma [7] is shown in Section 15. 1[ A key point in its proof is 
that the martingale term —{q, Cn+i) + 2(en+i)a5 iii Lemma[5l is a linear function 
of Cn+i and e„+i which do not depend on q, so that the two corresponding 
convergence results of these martingales will apply from any reference point q £ 
£s n AA(x). It will enable us to prove that, if r is a accumulation point of f(n), 
then Vr{v{n)) a.s. converges to if r G N{x) although r is random. 

Lemma 7 There exist eo := Cst(x,a) and ui := Cst(x,a) such that, if for some 
e ^ eo and n ^ ui, v{n) G Bv^{e/2), then 

V{C{BvAe)) I ^n) ^ l-exp(-e2Cst(x,a)(n + no)). 

Next, we provide in the following Lemma [8] some information on the rate of 
convergence of v{n) to f (oo), which will be necessary for the asymptotic estimates 
on the frontier Aq{v{oo)) in Lemma [TOl 

Lemma 8 There exist e, v := Cst(x,a) such that, a.s. on C{By^{e)), 

lim (f (n) — v{oo))n'^ = 0. 

n— ►oo 

The proof of Lemma [8l given in Section [5T2t starts with a preliminary estimate 
of the rate of convergence of H{v{n)) to H(v{oo)). To this end we make use of 
Lemma [9] below, giving the stochastic approximation equation of H{z{n)). It 
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implies, together with Lemma [6] (a), that the expected value of H{z{n + 1)) — 
H{z{n)) is at least Cst{x, a){H{x) — H{z{n)), so that we can then estimate the 
rate of H{v{n)) to H(x) by a one-dimensional technique. 

Finally, this estimate implies similar ones for the convergence of J{v{n)) and 
Ij;(oo)(t'(n)) to by Lemma El so that we conclude using entropy estimates for 
the rate of convergence of V^(^^^{z{n)), using again that only two martingales es- 
timates are necessary, given the linearity of the perturbation in ()17p with respect 
to the reference point q £ £s H AA(x). 

Lemma 9 For all n S N, 

H{z{n + 1)) - H{z{n)) = \ '^j^f^l + + s^+i, (20) 

n + uq + 1 H (v(n)) 

where E,{^j-i+i I ^n) = and, if for some a > 0, v{n) £ and n ^ Cst(a), then 

II ^ Cst(a,a, |G|) Cst(a,a, |G|) 

n + no (n + no)^ 



Lemma [9] is proved in Section [4.51 

The next Lemma [TU] yields the asymptotic behaviour on the border sites 
dS. This behaviour is similar to the one one would obtain without perturbation 
(i.e. with {en)nm* = in ([STj)). Indeed, if i £ dS, then Ni{x) - H{x) < is the 
eigenvalue of the Jacobian matrix of ^ in the direction {5i,j)j<=,v{G) (see the proof 
of Lemma [T]), and the renormalization in time is approximately in H{x)~^ logn 
(see equation ([37]) ). so that the replicator equation ([6|) would predict that i G dS 
is visited of the order of n^^^^^^^^^^ times at time n. This similarity with the 
noiseless case is due to the fact that the perturbation (e„)„gN* is weak near the 
boundary (see Lemma [4] (b)). 

Lemma 10 There exists e := Cst(x,a) such that, a.s. on C{By^{e)), ^^(^(oo)) 
occurs a.s. 

The proof of Lemma [TOl given in Section 15.31 niakes use of a martingale 
technique developed in [18j, Section 3.1, and in [7J in the context of strong edge 
reinforcement. We could have shown this Lemma [TOl by a thorough study of the 
border sites coordinates of the stochastic approximation equation (j37p . but it 
would lead to a significantly longer - and less intuitive - proof. 

Now we do not assume anymore that V{G) = T{x) for some x G A, in 
other words we let the graph (G, ~) be arbitrary, possibly infinite. The following 
Proposition m obviously implies Theorem [3l 

Let, for all n, /c G NU {oo}, n ^ /c, TZn,k be the range of the vertex-reinforced 
random walk between times n and A;, i.e. 

'T^n,k '■= {i £ G s.t. Xj = i for some j £ [n, k]}; 

note that, for all n G N, 7^ C ^. 
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Proposition 4 Let x € £s- There exists e := Cst(x,a) such that, for all n ^ 
Cst(x,a), if v{n) £ Bv^{e/2), then 

P({7^n,oo = T(x)}n/:(Sv/Je))n^a(t;(oo)) I ^„)>0. 

Moreover, the rate of convergence is at least reciprocally polynomial, i.e. there 
exists V := Cst(x,a) such that, a.s. on ll{By^{e)), 

lim (u(A;) — v{oo))k'^ = 0. 

fc— >oo 

Proposition S] is proved in Section [531 Observe that, if G = T{x), then it is 
a direct consequence of Lemmas U\ El and [101 The localization with positive 
probability in this subgraph T{x) results from a Borel-Cantelli type argument: 
the probability to visit dT[x) at time n starting from S{x) is, by Lemma [T0| 
upper bounded by a term smaller than n"~^, where a « ma-Ki^Qs Ni{x) / H (x) < 
1, and < oo. Technically, the proof is based on a comparison of 

the probability of arbitrary paths remaining in T{x) for the VRRWs defined 
respectively on the graphs T{x) and G. 

2.4 Contents 

Section [3l concerns the results on the deterministic replicator dynamics: Lemmas 
[IHSland Lemma [Hare proved, respectively, in Sections I3.1H3.3I and 13. 4[ 

Section [H develops the framework relating the behaviour of the vector of den- 
sity of occupation v{n) to the replicator equation ([6|): we write the stochastic 
approximation equation ()37p in Section [4. 11 establish in Section [4. 21 some prelimi- 
nary estimates on the underlying Markov Chain M{v), prove LemmaHlin Section 
14.31 prove Lemmas [3 and [3 (stochastic approximation equations for Vq{z{n)) and 
H{z{n))) and inclusions in Sections I4.4H4.51 

Section [5l is devoted to the proofs of the asymptotic results for the VRRW: 
Lemma [3 in Section 15.11 on the convergence of v{n) with positive probability. 
Lemma m in Section [5. 2 1 on the corresponding speed of convergence. Lemma [TOl in 
Section 15.31 on the asymptotic behaviour of the number of visits on the frontier of 
the trapping subset, and Proposition [H in Section [5?^ on localization with positive 
probability in the trapping subsets. 

Finally, we show in Appendix lA.ll a Lemma on the remainder of square- 
bounded martingales, which is useful in the proofs of Lemma [8l and Proposition 
[H whereas Appendix lA. 21 is devoted to the proof of this Proposition [H 

3 Results on the replicator dynamics 
3.1 Proof of Lemma [1] 

Note that DF{x)v = —H{x)v if S{v) n T{x) = 0, so that it is sufficient to study 
the eigenvalues of DF{x) on {v G M^^*^) s.t. S{v) C T(x)}; hence we can assume 
that V{G) is finite (equal to T[x)) w.l.o.g. 
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Let S := S{x) for convenience. For all i, j £ V{G), 



m 

dxj 



Ni{x) — H{x) if Xj = and j = i 

if Xj = and j ^ i 

Xi[aij — 2H{x)] if Xj / and xj ^ 

Xi[ai,j — 2Nj{x)] if Xj / and xj = 



Let us now consider matrix DF(x) by taking the following order on the indices: 
we take first the indices i, j £ V{G) \ S, and second the indices i, j S S: 

D\ag{Ni{x) - H{x)),^v(G)\s (0) 
(*) DB 

where 

B = [aij - 2H{x)]^ j^g , D = Diag(xi)i6s. 

The matrix DB is easily seen to be self-adjoint with respect to the scalar 
product {u,v)£)-i := {D^^u,v). Hence DB has real eigenvalues. This proves the 
first statement of the lemma. 

Remark that, if we consider ([6]) as a differential equation on R^^*^) then 

(^Fix), I) = ^^Mll) 1,^0^,(0)^, = -((x, I) - l)H{x). 

This implies, if x G A (so that (x, I) = 1), that, for all vector u S R^^*^), 

{DF{x)u, I) = -H{x){u, I). (21) 

Hence p : u ^ ("U, I) is an eigenvector of ^DF{x) with eigenvalue —H{x). This 
makes —H{x) an eigenvalue of DF{x) and, more precisely, 

Sp{DF{x)) = {-H{x)} U Sp(DF(x)|ta); 

indeed, by (I2ip . an eigenvector u of DF{x) with eigenvalue A 7^ —H{x) belongs 
to Ker p = TA. Therefore, the stability of an equilibrium x of ^ on R^^*^) is 
equivalent to the stability restricted on A, which completes the proof of the first 
equivalence in statement (b). 

Claim. Let M = Diag(yi, . . . ,yr) be a diagonal r x r matrix, with yi, . . . 
Ur G and let be a symmetric r x r matrix. Then minSp(A^) ^ <^=^ 
minSp(MA^) ^ and, under this assumption. 



minSp(MA^) ^ min Sp(A^) min{yj} 



l<i<r- 



Proof of the claim. It suffices to prove that minSp(A^) ^ implies 
minSp(MA^) ^ and the corresponding inequality, since the inverse statement 
is symmetrical. 
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Recall that, for any r x r symmetric matrix R with nonnegative eigenvalues, 
there exist a diagonal matrix D and orthogonal matrix Q such that R = Q^DQ, 
hence 

minSp(i?)= mi(Dt,t)= inf (DQt,Qt) = mf(Rt,t). 

mi mi mi 

Let us define L = Diag(^, . . . , Observe that = M. Now MN = 

L{LNL)L-^ implies Sp(M7V) = Sp{LNL). 
LNL is symmetric; therefore 

minSp(MiV) = minSp(LiVL) = inf {LNLt,t) = inf {NLt,Lt) 

^ inf (Nu,u) = min yj inf {Nu,u) = min yjSp(A^). 

□ 

To complete the proof of statement (b), we apply the claim to M := D and 
N := -B. 

It remains to prove that a stable equilibrium in A is feasible. Let x £ A 
be such an equilibrium. Assume that H{x) = 0. If Xj = for some i, then (by 
Lemma [T] (b)), Ni{x) = so that xj = for all j ~ i. Hence x = 0, which is 
contradictory. If now Xi ^ for all i, then G is necessarily finite (by definition 
of A), and a = {ai,j)ij£V{G) = since its eigenvalues are nonpositive (Lemma [T] 
(b) again) and its trace is nonnegative. This is again contradictory. 



3.2 Proof of Lemma [2] 

Let d := ds, (P) := (P)^ and (P)' := (P)'^ for simplicity. 

Assume (P) holds for some d ^ 1. Let us prove that, if i, j, k S are such 
that i ^ j k, then Ojj = aj^^. 

If i = j, then i = j ^ k implies, by (P)(a)-(b) that k ^ S - and therefore 
a contradiction - since if k were in S, it would be in the partition of i, which is 
a singleton. If i 7^ j 9^ /c, then j and k are in the same partition of S. Hence 
o-ij = CLi,k by (P)(c), which completes the proof of (P)'. 

Assume now (P)'. Let us prove that the relation R defined on S by 

iRj <^=^> i ^ j ox i = j 

is an equivalence relation on S. It is clearly symmetric and reflexive. Let us 
prove that it is transitive: let i, j, k £ S be such that iRj and jRk, and prove 
iRk. This is immediate if i = j or j = k; hence assume that i ^ j and j 7^ k; 
then (P)' implies ds{i} = ds{j} = ds{k}. If we had i ^ k, then it would imply 
k G ds{i} = ds{j}, and therefore j ~ k, which leads to a contradiction. 

Now let us prove that there is only one element in the partition of a loop. 
Assume that iRj, i ^ i and j ^ i for i, j £ S; (P)' implies in this case that 
= CLij > 0, so that i ~ j, hence i = j since iRj holds, which leads to a 
contradiction. 
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Let Vi, i = 1, ... d be the partitions of R: elements of different partitions 
are connected, by definition, and (P)(a)-(b) holds for some 1. Let us prove 
(P)(c): let i, j £ {1, . . . , d} be such that i ^ j, and assume a £ Vi, (3 £ Vj. Let 

Wa,f3 := {{a',f3') G S"^ s.t. a^',/?' = ffla,/?}. 

By applying (P)' twice, we firstly obtain that Wa,f3 5 {a} x Vj, and secondly 
that Wa,/3 ^Vi X Vj, which enables us to conclude. 



3.3 Proof of Lemma [3] 

Let S := S{x) and (P)' := (PYs^x) simplicity. Let 

B = [aij - 2H{x\^^s ' 
and maxSp(S) ^ Vt G M^,S[t] ^ 0. Observe that, for all t = {ti)i^s e 

B[t] = - ^H{x))Utj = H{t) - 2H{x) (^.t^ . 

i,jes \ies J 

Let us assume that (P)' does not hold, and deduce that B[t] > for some t G M"^, 
which will prove the first statement. 

There exist i, j, k £ S such that j k and ajj 7^ Oj^fc (otherwise (P)' would 
be satisfied). Let, for all A G M, 

tx ■■= {t{v=^} + AI{^=j} - (1 + A)I|„=fc})^es G M^, 

then 

B[tx] > 2\{ai,j - ai^k) - 2ai^k, 

so that B[t\\ > for some A G M, which yields the contradiction. 

Let us now assume that (P)' holds, and that ajj = clj^j, with c = 1 for 
simplicity. First assume S contains no loop. Then, by Lemma [21 5 is a d-partite 
subgraph for some d^ 1 ((P)^(a) holds); let Vi, ... be its partitions, then 

B[t] = {h^j - 2H{x))titj = -2H{x) ^i-M] 
ij&s Vies / i,j&s 

=-2H{x)(Yvk) +(Ev)i -x:^ 

\fc=i / \fc=i / k=i 



where, for alH G {1, . . . , d}, Vk = Sigyfe Therefore 



k 

2 



B[t] = -i2H{x)-l)(YA -Y4^0, 

\k=l / k=l 
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where we use the fact that H{x) ^ 1/2, since H{x) = 1 — 1/d and d ^ 2 (see 
proof of Theorem [21 Section ITTT^i . 

Now assume that S contains one loop; then, again by the proof of Theorem 
El Section [2.2.21 it is a chque of loops and H{x) = 1; thus 

B[t] = -2(Y.A +fE*^l =-fe'^l 

Vies / Vies / Vies / 

3.4 Proof of Lemma [6] 

Let us first prove (a) in the case q := x, which will imply H{q) = H{x) for any 
equilibrium q S N{x) and therefore imply (a) in the general case. Let x ^ Eg, 
and let y G TA be such that x + y G A. Let S := S{x) for simplicity. 
Then 

H{x + y)= a^j{x^ + y^)ixj+yj)=H{x) + 2 Ni{x)yi + H{y) 

«je V{G) iey(G) 

(22) 

= H{x) + 2 Y {N,{x)-H{x))y,+ ^ («„• - 2/7(x))y,y, 
ieV{G) ijeV^(G) 

= H{x) + 2 Y (iV^(a;)-^(x))y.+ J^(a„-2i7(x))y,y, (23) 

i6V(G)\S ijeS 

ieV(G)\S 

^H{x) + 2 Y {N^{x)-H{x))y,+ ^ ^^(y) 
ieV{G)\s iey(G)\s 

In the third equality, we make use of the identity J2ieV(G) Vi — ^i whereas in 
the fourth equality we notice that Ni{x) = H[x) for all i G and that the 
reinforcement matrix a := {ai.j)i,j^y(^G) is symmetric, and let 

Wi{v) ■■= Vi 2^(aij - 2H{x))yj + ^ (oij - 2H{x))yj = oiyi^oiVi) 

V j6V(G)\S / 

= O|j/Ho(y9s), 

using that, for all j £ dS, yj ^ 0. Finally, we apply in the inequality that 
B := (aij — 2H{x))i^j(zs is a negative semidefinite matrix. 

Using that, for all i G dS, Ni{x) < H{x) (and yi ^ 0), we deduce that 
there exists a neighbourhood J\f{x) of x in A such that, \i x + y G M{x), then 
H{x + y) ^ H{x). 

In order to obtain the required estimate of H{x + y) — H{x) we observe that, 
if z := {y-i)ies then, by semi-definiteness of B symmetric, 

- Cst(x, a)\Bz\'^ ^ {Bz, z) = ^ " '^H{x))yiyj ^ -Cst(x, a)\Bz\^. (24) 

ijes 
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But 




= {N,{y)+2H{x)yas)i(,s 



where we use that ygs = —ys in the second equahty, since y £ TA. Hence 

\Bzf = Y^i^iiy) + ^H{x)y9sf = N,{yf + o\y\^Q{y9s) (25) 

and, if we let 

K{y) ■.= Y,Ni{yf+yds 

ies 

then, by combining identities ([23|) . (p^ and (p5]) (and using that t(;i(y) = 0|j^|^o(2/95) 
for all i E 9S) , restricting J\f{x) if necessary, 

- Cst(x, a)K{y) ^ F(x + y) - i/(x) ^ -Cst(x, a)K{y). (26) 

On the other hand, let 

Then, again by restricting J\f{x) if necessary, 

Cst(x, a)L{y) ^ J(x + y) < Cst(x, a)L{y), (27) 
where we use again that Ni{x) < H{x) for all i G 55. But 

L(y) = Y^my) - {H{x + y) - H{x))f + yes 

= K{y) + 0|,|^o(l^(^ + y)- H{x)\). (28) 

Combining inequalities (|26p . (|27p and (j28p . and further restricting J\f{x) if 
necessary, we obtain inequality (fTSj) as required. 

Let us now prove (b). If g G S{S{x)) and y E A, then 

and 

ieS" ieG iGG i&dS 

where we use that {aij)ij(zG is symmetric in the second equality, and that q is 
an equilibrium in the third equality. Therefore 

I,{y) = H{y) - H{q) + ^ yi[2{N,{y) - H{y)) - {Ni{q) - H{q))]. (29) 

ieds 

If q, y £ M{x) then, by restricting M{x) if necessary, x £ Eg implies that for all 
i G dS, 

-Cst(x, a) ^ 2{Ni{y) - H{y)) - {Ni{q) - H{q)) ^ -Cst(x, a). 
Inequality (fT9|) follows. 
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4 Stochastic approximation results for the 
VRRW 



4.1 The stochastic approximation equation 

The main idea is to modify the density of occupation measure 

Zn{v) 



v[n] 



into a vector z{n) that takes into account the position of the random walk, so 
that the conditional expectation of z{n + 1) — z{n) roughly only depends on 
z[n) and not on the position X„. This expectation will actually approximately 
be F(z{n))/(n + no), where F is the map involved in the ordinary differential 
equation ([6]). 

For all X € A, let M(x) be the reversible Markov Chain with transition 
probabilities 

M(x)(^,i):I..,^^^M^. (30) 

Note that M{v{n)) provides the transition probabilities from the VRRW at time 
n. Recall that 7r(x) in ([2]) is the invariant probability measure for M(x). 

Let us denote by G (resp. TC) the set of functions on V{G) taking values in M 
(resp. in R*^). Let I be the function identically equal to 1. Let M{x) and n(a;) 
denote the linear transformations on Q defined by 

{M{x)m):=Y,M{x){i,j)f{j) (31) 

n(x)(/):= (Y.7r{x){i)f{i)] I. (32) 

Note that, by a slight abuse of notation, M(x) equally denotes the Markov Chain 
defined in ()30p and its transfer operator in (jSip : Il{x) is the linear transformation 
of G that maps / to the linear form identically equal to the mean of / under the 
invariant probability measure it{x). 

Any linear transformation P of G (and in particular M(x) and n(j;)) also 
defines a linear transformation of Ti: for all / = {fi)i£G £ "H, 

Pf := {PfiUa. (33) 

Let us now introduce a solution of the Poisson equation for the Markov Chain 
M{x). Let us define, for all t G M+, 



oo 



fM{x) 
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which is the Markov operator of the continuous time Markov Chain associated 
with M(x). For all x G Int{A), M{x) is indecomposable so that Gt{x) converges 
towards n(x) at an exponential rate, hence 



/•oo 

Q{x) := / {Gt{x)-Ii{x))dt 
Jo 



is well defined. Note that 

Q(x)I = 0, 

and that Q{x) is the solution of the Poisson equation 

(/ - M{x))Q{x) = Q{x){I - M{x)) =1- n(x), (34) 

using that M{x)n{x)f = n(x)/ = n(x)M(x)/ for all / G ^ (or / e H). 

Let us now expand v{n + 1) — v{n), using Let {ei)i^G be the canonical 
basis of MP , i.e. := (Ij=j)jgG for all i ^ G. Let t G W be defined by 

l: G — 
i I — > ej. 

By definition, 



n + no + 1 /jgG V n + no + ly n + no + 1 

so that, using that Ii{x)i = 7r(a;)I for all x G A, 

(n + no + l)(u(n + 1) - v{n)) = L{Xn+i) - v{n) 

= [I- n{v{n))]i{Xn+i) + {T^{v{n)) - v{n)) 
= [I -Yi{v{n))]i{Xn+i) + F{v{n)), 

where F is the function in definition ([5]). 

Now, 

[/ - Il{v{n))]i{Xn+i) {Q{v{n)) - M{v{n)Q{v{n)))i{Xn+i) 



where 



n + no + 1 n + no + 1 

= Cn+l + Tln+l + rn+l,l + r„+i,2, (35) 



Q{v{n))i{Xn+i) - M{v{n))Q{v{n))i{Xn) 
n + no + 1 



f 1 1 A ,w . . ^^ , . M (v (n))Q (v (n)) dXn) 

rn+1,1 := — — - —— M{v{n))Q{v{n))i{Xn) = - \_ ^ "/ 

V^' + '^o + l n + no/ (n + no)(n + no + 1) 



M{v{n))Q(v[n)),{Xn) M{v{n + l))Q(7;(n + l))i(^n+i; 



n + no n + no + 1 

[M(t;(n + l))Q{v{n + 1)) - M{v{n))Q{v{n)MXn+i) 
n + no + 1 
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Let, for all n e N, 



.(n):=,(,.) + ^""'"»g'-'"''"^"'. (36) 

n + no 



and 

1 F{v{n)) - F{z{n)) 
""''^'^ n + no + 1 H{v{n)) 

rn+l ■= rn+1,1 + rn+1,2 + rn+1,3- 



Then, for all n G N, it follows from equation (j35|) that 

1 F(z(n)) 

z{n + 1) = z{n) + — — ) ) \\ +en+i + rn+i. (37) 

n + no + 1 {V[n)) 

Note that E(e„+i | = 0, since E(Q(i'(n))/,(X„+i) | = M(f (n))Q(z;(n))i(X„); 
also observe that 

iey(G) iey(G) + f^o 

We provide in Section 14.21 estimates of the conditional variance of e^+i and of 
r„+i, which will be sufficient to prove localization of the vertex-reinforced random 
walk with positive probability. 

4.2 Estimates on the underlying Markov Chain M{v) 

For convenience we assume here that V{G) = S U dS, where {S, ~) is finite, 
connected, and not a singleton unless it is a loop. Let a := maxij^ci^j o-ijy 
a := minjjgci-i Oij - 

Let us first introduce some general notation on Markov Chains. Let K be 
a reversible Markov Chain on the graph (G, ~), with invariant measure /x. Let 
< ., . >^ be the scalar product defined by, for all f, g £ Q, 

< f,g >ti-= ^ f{x)g{x)^i{x). 

xeG 

On Q, we define the (P{iJi) norm, 1 ^ p < oo by 

11/11^^^)^= 1/(^)1^1 ' 

\x&G J 

and the infinity norm 



, := max I f{x)\. 

xeG 

We also define the infinity norm on TL: if / = {fi)ieG G 



oo = max ||/i||oo = max (38) 

iSG i,x£G 
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Let be the expectation operator 

where I is the constant function equal to 1. 
We let £k be the Dirichlet form of K 

£K{f,g) =< {l-K)f,g>^, 

and let Var^ be the variance operator 

Var^(/) := 11/ - E,f\\%^^^ = \\f\\%^^^ - (E^/)^. 

Simple calculations yield that 

and 

Var;.(/) = ^ j;(/«-/(i)))V(iMi)- 
Let \{K) be the spectral gap of the Markov Chain K, 



\{K) := min 



The following Lemma [TT] states that the spectral gap of the Markov Chain 
M{v) is lower bounded on Aq, (defined in (|14|) ). 

Lemma 11 For all v £ A^, X{M{v)) ^ Cst(a, a, |G|). 

PROOF: Let M := M{v) and vr := tt(v) for simplicity. Let us first observe that, 
for all i £ G, j G S such that i ^ j, 

^ avj/a ^ aa/a and M{i,j)TT{i) = TT{j)M{j,i) ^ aa^liizs/a, (39) 

where the second inequality comes from 

.s ... aijVj ViNi{v) ai^jViVj op? 
M{i,j)7r{i) = \ , . = -^T-y- ^ —lies- 
Ni(v) H{v) H{v) a 

Now, by connectedness of {S, ~'), for all i, j £ G, there exists / ^ |G| and a 
path (nfc)i^fc^/ G V{G) x 5"'^^ x V{G) such that i = ni, j = ni, Uk ~ Uk+i for 
all A: G {1,...,/- 1}. 
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Hence, for all A; G {1, . . . , /}, using inequalities ([39|) . 

fce{i,. ..,«-!} 

^ lnii){f{i) - f{n2)f + InimU) - f{m^i)f + I Yl (^(^'^) " f(^k+i)f 

k(^{2,...,l-2} 

^ ^[M(i,n2)7r(i)(/(i) - fin2)f + M{j,ni_^)^{j){f{j) - /(n,_i))'] 
aa 

+ —J {f{nk)-f{nk+i)fM{nk,nk+i)ir{nk) 

- ke{2,...,l-2} 

- fce{i,. ..,/-!} - 
Therefore 

Var.(/) = ^ 5] - /(j)))' ^ ^SmUJ). 

□ 

The following Lemma[T2]provides upper bounds on the norms oiQ{v), M[v)Q[v) 
and their partial derivatives on A^, which will be needed in the estimates of r^+i 
and of the conditional variance of e„+i in Lemma HI 

The norm on linear transformations of Q will be the infinity norm 

ll^ll - SUP 

Halloo •— sup .. .. 

/egj^^o llJlloo 

Note that, for any linear transformation A of Q, the corresponding linear trans- 
formation of Ti (still called A) defined in (j33p still has the same infinity norm 
(the ||.|| oo on H is defined by (j38p ) 



|A|U= sup ll-^^ll- 



fenj^o ii/iioo 



Lemma 12 For all v£Aa,i, j£G,f£G, 

(a)MM(i,i)<(^)'^ 

V a / 



(b) ||Q(.)/||,.(.(.)) ^ ^ A(M(.)) 

(c) ||Q(t;)||oo ^ Cst(a,a,|G|), \\M{v)Q{v)\\oo ^ Cst(a,a,|G|) 

(d) 11-^— lloo ^ Cst(a,a, |G|), || — ||oo ^ Cst(a, a, 
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proof: Let M := M{v), Q := Q{v), vr := 7r(t;), A := X{M{v)) for simplicity. 
Inequality (a) is obvious: for all j £ G, 

M(i = = ^j^i(^) a^,jH{v) /aV Trjj) 

^ Ni{v) H{v) Ni{v)Nj{v) " \aj ' 

Let us now prove (b). For all / G ^, 

l|GJ-^(/)||,2.(,)^e"2^*Var.(/), 
by definition of the spectral gap (see for instance Lemma 2.1.4, [17]), so that 



\\Q{y)fyM < 



iGtiv)f-U{v)f)dt 



^ / \\iGt{v)f-U{v)f)\\,,^^^ dt 

£2(7r) JO 



/•CO 

Jo 



A 



A 



(40) 



Inequality (c) translates this upper bound of the ^^(vr) £^(7r)-norm of Q{v) 
into one involving the infinity norm for MQ, using (a): 



\MQf{z) 



Y,M{i,j)Qfij) 



1 /-\ 2 

1 / a 



\\Qf\\e^{n) 



^ .a\^ WQfWpjn) ^ / « 



Hence, using Lemma [TTl 



WMQfWoo < 



— \ 2 

a 



a J Xa^ 



^2(7r) ^ / a 



^2(7r) 



a j \a- 



Aq2 



^Cst(a,a,|G|)||/||oo. 



Then the same upper bound for ||Q(f)/||oo follows from the Poisson equation 
(IMD: 

Q(y) = M(v)Q(v) ^ 1 - Xi(v). 

Let us now prove (d). Given i £ G, let us take the derivative of the Poisson 
equation Q{v){I — M{v)) = I — Il{v) with respect to Vi'. 



dvi 



dvi 



dvi 



This equality, multiplied on the right by Q{v), yields, using now the Poisson 
equation (/ - M{v))Q{v) = I - U{v), 



dvi 



dvi 



dvi 



dvi 
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where we use that, for all / € ^, 
dQ{v) 



dvi 



dO(v) 

n(^;)/ =< /, I -^1 = 0, 



since Q{v)l = for all v G A. 

The equality (j^T|) implies t 
following estimates hold: for all i, j, k £ G, j ^ k, 



The equality (j^T|) implies the required upper bound of ||^^j^||oo- Indeed, the 



d[M{v){j,k)] 








dvk ttj^k 


Uj^kVk dNj{v) 


dvi 




dvi \Nj{v)) 




dvi Nj{v) 


Nj{vf dvi 



^ 2a ^ 2a 
^ Nj{v) ^ aa' 

where we use that aj^kVk ^ ^i(^) dNj /dvi{v) = aj^i, and that there exists 
I € S with I ^ j, given the assumptions on S. Also, 



d7r{v){j) 




dvi 





d fvjNj{v) 



dvi \ H{v) 



d{vjNj{v)) 1 VjNj{v)dH{v) 



dvi 



H{v) H{v) 



dvi 



^ 4a ^ 4a 



2Ni{v) ^ 2a. The upper bound of 



I diM{v)Q{v)) I 



I oo 
□ 



where we note that I I 
follows directly. 

4.3 Proof of Lemma [4] 

The estimates (a) and (d) readily follow from the definitions of e^+i and z{n), 
and Lemma [T2] (c). 

Let M := M{v{n)), Q := Q{v{n)), vr := 7r(u(n)), A := \{M{v{n))) for sim- 
plicity. Let us prove (b): 



n 



1 /-\ 2 

1 / a 



\ a 



i6G 



a \ 1 



a I a 



WQe 



^ Cst(a,a, |G|)||ei||^2(^(^(„)) ^ Cst(a, a, |G|)t;(n)i, 

where we use Lemma [T2] (a) and (b) respectively in the second and in the third 
inequality. 

In order to prove (c), let us first upper bound ||r„+ii||oo using Lemma WI\ 
(c): 

„ „ ^ \\M{v{n))Q{v{n))i{Xn)\\oo ^ Cst(a,a,|G|) 

lFn+l,l||oo ^ ■ ^ 



(n + no) 



(n + no) 
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Let us now bound ||r„_|_i_2| 



(n + no)||r„+i,2||oo ^ sup 
ee[o,i] 



^'S2\iv{n + 1) - v{n))i\ sup 

ieG, ee[o,i] 



d{MQ){ev{n) + (1 - e)v{n + 1)) 



Cst(a,o, |G|) 
n + no 



de 



d{MQ){ev{n) + (1 - + 1)) 



dvi 



where we use Lemma [12] (d) in the last inequahty. 

It remains to upper bound ||r„+i^3||oo- First observe that, for all y = {yi)i,^G, 
z = {zi)i^G & A, i £ G, 



\Fi{z) - Fi{y)\ ^ J2 - Vjl sup 



fceG, xeA 



dFi\ 



X 



dxk 



^ 2a^ |zi - yi\, 
ieG 



where we use the explicit computations of dFi / dxj in the proof of Lemma [TJ 
Hence 

\\F{z)-F{y)\\^^2a\G\\\z-y\\^, 



which implies 



kn+l,3||oo ^ 



1 |G| 



2a\G\\\v{n) - z{n)\\oo ^ 



Cst(a,o, |G|) 



where we use that, by inequality dS]), H{x) ^ a/\G\ for all x G A. 



4.4 Proof of Lemma [5] and inclusions ( 1151) 



Let us first prove inclusions ()15l) . If we let g : M+ \ {0} — > be the function 
defined by g{u) := u — log(u + 1), nonnegative by concavity of the log function, 
then, for all y G A such that yi > for all i £ S, 



= - E 9^ + + 2^95 = E ( ^^^-^ ) + 32/95, (42) 

iG5 ^ 9^ ^ ies 



which implies the inclusions. 

Let us now prove Lemma [5l let, for all n G N, 



Cn 



+1 



with the convention that Cn+i = if z{n)i = for some i £ S. Fix e > such 
that -6^,(26) C Aq, for some a = Cst{q) > 0, and assume v{n) G -Bvg(e) for some 
n ^ ni. Thus \\z{n) — 'i^(^)||oo ^ Cst(Q', -\- no) by Lemma 13 (d) ; we assume 

in the rest of the proof that e < Cst(g) and no ^ C5t{q,a) so that, using (|12|) . 
^(n) G r2e) C A„. 
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Note that \\v{n) — v{n + l)||oo ^ (n + no) ^, which imphes, using LemmaSl 
that \\z(n) — z(n+l)||oo ^ Cst{q, a){n -\- uq)^^ . Hence, using that z{n) G Aq, 

+ 1)) - V,{z{^)) = -Y.q^ log f^t-t^) + 2[z{n + 1)95 - z{n)9s] 

Ezfn + l)j — 2;(n)j , , , , . . ( Cstfg, a)\ 

where we again make use of notation u = ^{y) <^=^ \u\ ^ v from Section [2.11 

Hence, using identity (137]) and Lemma H] (b), we obtain subsequently (recall 
that is defined in (fT6]) l 



y,(z(n + 1)) - V,{z{n)) = , V ^ ^'^''^''W - (g, C 



n + no + 1 H{v{n)) 
+ 2(e„+i)a5 + □ 



n+l; 



Cst((7, a) 

(n + no)2 



4.5 Proof of Lemma [9] 

Using identities ()22p and (I37p (recall that J is defined in ([7])), 

//(^(n + 1)) - H{z{n)) = 2 ^ iV,(z(n)).(z(n + 1) - z{n))i + H{z{n + 1) - z{n)) 

ieG 

n + no + 1 H{v{n)) ~^ '^""''^ ~'~ 

where 

:=2^iVi(z(n))(e„+i)i 

:= 2^ iVi(z(n))(r„+i)i + H{z{n + 1) - z(n)). 

Let a > 0, and assume u (n) E A^. Inequalities (l)-(2) follow from Lemmad] 
(a)-(c), and from \\z{n + 1) — ^;(n)||oo ^ Cst(a, a, \G\)/{n + no) (see for instance 
the beginning of the proof of Lemma [SJ. 



5 Asymptotic results for the VRRW 
5.1 Proof of Lemma \7\ 

Fix e > such that By^ie) C Aq, for some a > depending on x, and assume 
v{n) £ By^{e/2) for some n ^ ni. 

Let us define the martingales {Ak)k^ri, {Bk)k^n and {Kk)k^n by 

k k 
j=n+l j=n+l 

Kk ■■= -iq,Ak) + 2Bk, 
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with the convention that A„ := and = Hn ■= 0. Using Lemma [5] (a), it 
fohows from Doob's convergence theorem that {Ak)k^n, {Bk)k'^n and {Kk)k^n 
converge a.s. and in C^. The upper bound 1^;^ — Kk-i\ ^ r/(A; + ng) a.s., for 
some r := Cst(x, a), imphes that, for all fc ^ n + 1 and G M, 

,'r2 

E(exp(6'(«;fc - Kk-i) I ^k-i) ^ exp 



2 (fc + no)2 

On the other hand, (exp(0Kjt))fc^„ is a submartingale since {Kk)k^n is a martin- 
gale, so that Doob's submartingale inequality implies, for all 9 > 0, 

P ( supKfe ^ c I ) = P ( supe'''''^ ^e^" \J^n] ^ e"^"E(e^'"- | 

^ exp — PC + 



2(n + no^ 



Choosing 9 := c{n + no)/r^ yields 



Let 

T := < sup Kk < 

k^n 



C2 



sup Kfc ^ c I J^„ ) ^ exp { -^("- + "-o) ) • (43) 



12 



inequality (|13|) implies that 

P(T I JS^) ^ 1 - exp(-e^Cst(x, a){n + no)). 

Now assume that T holds, and let T be the stopping time 

T := inf{fc ^ n s.t. Vr(z(fc)) ^ 2e/3}. 

Note that, using Lemma[4](d), if n ^ Cst(a;, a), then for all fc G [n, T), T4(u(fc)) < 
e. We upper bound Vx(f (T)) — Vx{v{k)) by adding up identity p!7|) in Lemma 
[5] with g := X, from time n to T — 1: this yields, together with Lemma (U that 
Vx{z{T)) < 2e/3 if T < oo, if we assume n ^ ni := Cst(x,a) large enough and 
e < eo := Cst(x,a) small enough. 

Therefore Vx{v{k)) < e for all k ^ n. Using again inequality (fTTll . we obtain 
subsequently that 

liminf H{x) - H{v{k)) + v{k)ss = a.s. 

k-^oo 

since, otherwise, the convergence of (k^) as fc — > oo would imply lim^^oo Kc(-z(fc)) = 
limfc^oo yx{v{k)) = — oo, which is in contradiction with Vx{v{k)) ^ 0. 
Hence, there exists a (random) increasing sequence {jk)k^o such that 

lim H{v{ik)) = H{x), lim v{jk)ds = 0. 

fc— »00 fc— >CXD 
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Let r be an accumulation point of (jfc))fc^o- Tlien H{r) = H{x) and rgs = 0. 

Note that Vx{r) = lim^^^oo ^'^^(^(jfc)) ^ e. By possibly choosing a smaller 
eo := Cst(x,a), we obtain by Lemma [6] that r is an equilibrium, and by Lemma 
[Uthat it is strictly stable. 

Let, for all j G N, 



Aj := < sup \Ak — Aj\ < 



24 



}n{-Pi«.-B,i<^}. 



There exists a.s. j G N such that Aj holds; let Iq (which is random, and is 
not a stopping time) be such a j. 

Let A; S N be such that jk ^ Iq and Vr{z{jk)) < e/2. Then Lemma [5] applies 
to r G Sn£s and a similar argument as previously shows that, for all j' ^ j ^ jk, 
Vr{v{j)) ^ e and 

Vr{z{j')) ^ Vr{z{j)) + sup \Ak - Aj\ + 2 sup \Bk -Bj\ + , (44) 

if ni := Cst(x,a) was chosen sufficiently large. 
Now, liminfj_+oo ^•(•z(j)) = and 

lim sup \Ak — Aj\ = lim sup {B^ — Bj\ = lim ^^^^^"^ = 0, 

hence limj^oo ^•(^(i)) = which implies limj^oo ''^(i) = ^ and completes the 
proof. 



5.2 Proof of Lemma [8] 

Let us start with an estimate of the rate of convergence of H{z(n)) to H{x). Let, 
for all n G N, 

Xn:=H{x)-Hiz{n)), u^:=-JM^^ 

H{v{n))xn 

with the convention that i/„ := if Xn = 0. 

By Lemma [6] there exist e, A, /i := Cst(j;, a) such that, for all n G N such that 
v{n) G By^{2e), Vn G [A,^]. On the other hand, for all n G N, using Lemma [9] 
and the observation that J{z{n)) = if Xn = by Lemma [6l 



Xn+l — 1 ; — r Xn - Cn+l - Sn+1 

V n + no + 1 / 
^ ( 1 - T ) ^« - + (45) 

where 

s^+i := -Sn+i + (fn - A) max(-Xn, 0)/ (n + no + 1). 
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If v{n) G Bv^{2e) for sufficiently small e := Cst(x, o) then, by Lemma [9l 



4n+l oo ^ ■ , oo ^ 7 — ■ (46) 

n + UQ ^ (n + no) 



where we use in the second inequality that max(— Xn> 0) ^ Cst(x, a)/(n + no + 1), 
since \\v{n) — z(n)||oo ^ Cst(x, a)/(n + no + 1) by LemmaH] (d), and H{v{n)) ^ 
H{x) by Lemma [6l 
Let, for all n E N, 

A 



:= n (l 

k=l ^ 



n + no 



Note that Pn^'^ converges to a positive limit. Inequality ([^5]) implies by induction 
that, for all n G N, 

Assume C{Bv^ (e)) holds so that, in particular, T;(n) G ^{By^ (2e)) for large n G 
N. The upper bounds (f46]) yield, assuming w.l.o.g. A < 1/2, that Yl]=i^'j/I^j < 
oo and lE(^|)//5? < oo; the latter implies, by Doob convergence theorem in 

that Y^^=i^j/l^j converges a.s. Therefore Xnn^ is bounded a.s. 

We deduce subsequently, by LemmaE](a), that for all A ^ Cst(j;, a), J{v{n))n^ 
converges a.s. to 0, so that lim„^oo v{n)Qsn^ = in particular. This implies that 
lim„^oo Iv{oo){v{'n))n^ = by Lemma [6] (b). 

Now apply Lemma[5]with q := v{oo): for large n G N, 

K(^)(.(n)) = -f ^^M^^ f; a) -2 f; (e.),, 

k=n \ fc=n+l / k=n+l 



/ oo ^ 

+ Cst(x,a)n ( y^TT^ 
: o(n~'^) a.s.. 



if we still assume w.l.o.g. A < 1/2, so that YlT=n+i(^k)ds = o(n ^) a.s by 
Lemmas [4] (a) and lA.ll This completes the proof of the Lemma, using (j42p . 



5.3 Proof of Lemma [10 



Let, for all n G N and i, j G G, z ~ j, 



Then, by definition of the vertex-reinforced random walk. 
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is a martingale, and 



so that, by Doob convergence theorem in M^'-' converges a.s. 
Hence, for all i E 95, 

using Lemma [21 the symmetry of a and Nj{v{oo)) 7^ for all j £ G = T{x) 
in the third equivalence, and H{v{oo)) = Nj{v{(Xi)) for all j G 5" in the fourth 
equivalence {v{oo) being an equilibrium). 

5.4 Proof of Proposition [4] 

We can assume w.l.o.g. that X„ G T{x). First recall that, if G = T{x), then 
the proposition is a consequence of Lemmas [71 [8] and [TOl We will now compare 
the probability of arbitrary paths remaining in T{x) for the VRRWs defined 
respectively on the graphs T{x) and G. 

Let us introduce some notation. For ah A: e N and AQV{G), let := 
be the set of infinite sequences taking values in A, and let Tjf be the smallest 
fj-field on that contains the cylinders 

Cv,k ■= {"w ^ 'P^ s.t. Wo = vo,. ■ ■ ,Wk = Vk}, V e A^. 

Let := Vfeg^T^"^. Finally, let (X^)jgp^ be the random walk restricted to 
remain in the subgraph A after time n. 
For all k ^ n and v S T(x)'^, 

p((x„+i, . . . = I = P((x2.^ , . . . = ^ I .^„)yij, 

where 

.- 11 11 1 - Mx,=a} e (0, 1), (48) 

and denotes the value of Yn^k at . . . ,Xfc) := where Zj{w), w G 

y(G), n ^ j ^ k — 1, assumes the corresponding number of visits of X to w. 
This enables us to prove the following claim. 
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Claim For all E G r^(-), P((X,+„),eN G ^ | = IE(I,^tw. ^„y„,oo I ^„). 

Let us first prove the claim in the case E = cj^^'* = C^jf^^ H {Tln,oo ^ T{x)}, 
for some A; € N and v G T{x)^ . Indeed, we deduce from ()48p that, for all / ^ n, 



so that 



P((Xj-+„)jeN £E\J^n)= lim E(I tm. ^^Fn,; | Tn) 

= E(I T(a;). ^ pYn, oo \ ^n) 

where l^^oo := hm/^oo The claim follows by uniqueness of extension of finite 
measures on vr-systems. 

We now apply the claim for E := {7^n,oo = T{x)}riC{Bv^{€))riAd{v{oo)) and 
prove that, a.s. on E, Yn^oo > 0, which will complete the proof of the proposition: 
for all a £ dS{x), a.s. on E, if e is sufficiently small, then 

I{x.="} _^Z,{a)-Z,^,{a) 



< CXD 



where we use that, since Ad{v{oo)) holds, v{j)a ~j^oo Cj^'^^'"^'^^^^^^^^ ^ for 

some random C > 0, so that j(jv^(^('j))2 ~j^oo — , and Naiv{oo)) < 

H{v{oo)) = H{x) is e is sufficiently small. 



A Appendix 

A.l Remainder of square-bounded martingales 

The following Lemma provides an almost sure estimate of M„ — M^o for large n, 
when Mn is a martingale bounded in L'^{Q,J^,F). 

Lemma A.l Let (M„)„^o be a bounded martingale in L? , and let f : 

be a nondecreasing function such that {f (x))"'^ dx < oo. Then 

Mn-M^ = o(/(E((M„ - M^f)) a.s. 
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proof: For all n ^ 0, let Sn := E((M„ - Moo)^) and let 
Then, for all n ^ 0, 

Therefore (M„)„^o and {Nn)n'^o are martingales bounded in L^, and thus con- 
verges a.s. 

Now, letting 0„ := Nn — for all n ^ 0, 

00 00 
M„ - Moo = ^ /(sfc)(Ofc - Ofc+i) = /(s„)0„ + if(^k) - fisk~i))Ok 

k=n k=n+l 

= o{f{sn)) a.s. 

□ 

A. 2 Proof of Proposition [1] 

Assume Xq := for simplicity. Let, for all n G N, 

Ar, := Zni-1) + ^n(-l), «n := ^n(±l)Mn, 

i?„ := Z„(0)M„ - log^„, 5„ := log ) = log 



Let a e (0, 1), e < [a A (1 - a)]/2. Given no G N with 2'„o(0) sufficiently large 
andX„o = 0, assume that Z„o(-2)/ log Z„,(2)/log ^^^(l) G (1/3,1/2), 

a^^ £ (a— e/3, a+e/3) and G (—e/3, e/3), which trivially occurs with positive 
probability. 

Let us define the following stopping times 



Tf 



T 



inf{n ^ no s.t. X„ G {-3,3} or X„ = X„„2 e {-2,2}}, 

inf{n ^ no s.t. Z„(2) V Z„(-2) > logZ„(0)}, 

inf{n ^ no s.t. a" (a - e/2, a + e/2) or Rn (-e/2, e/2)}. 
To ATi ATs. 



For all n G N, let tn be the n-th return time to 0, and let t'^ := t„ A T. 

It follows from elementary estimates that, as long as no ^ tn < T, for suf- 
ficiently large ^tio(O), ^t„(-l) G ((a - e)n/logn, (a + e)n/logn) and Zt„{l) G 
((1 — a — e)n/ log n, (1 — a + e)n/ log n). 

We successively upper bound P(To < Ti AT2 | J^-no), P(Ti < Tq AT2 | .F^q) and 
P(T2 < To A Ti I .Fno), which will enable us to conclude that F(T = oo\J^no) > 
for large ^no(O). 



40 



First, for sufficiently large Z„(,(0), 
P(To <Ti ATs I Tn,,) 

^ nXt^+A = ^t„+2 = ±2 I J^n,) + F(Xt„+3 = ±3 I 

^Cst(a,e) E ^<^- (49) 

n>Z„o (0) 

Let G := (.^t;j)n^z„o(o)) and let us consider the Doob decompositions of the 
G-adapted processes Rf^ and St'^^ n ^ Z„q(0): 

Rt'^ = Rno + A„ + ^n, 
St'„ := Sn,) + <!>„ + 

where A^^jo) = ^z^jo) = ^z„,(o) = ^z„o(o) := and, for all n > Z„(,(0), 
A, - A„_i := E{Sn - Sn-l I $n " $n-l := E(T„ - r„_i I 

and (^'n)n^z„o(o) and (H„)„^2^^(o) are G-adapted martingales. 
Easy computations yield that, for all n ^ Z„o(0), 

|A„-A„_i| ^Cst^i^^, |cI>„-cI>„_iKCst(a,e)fi^y, (50) 



(logn)4 = ^2 I ^ ^ ^ ,^ /^log^^^ 

(51) 



E((^„+i-^'„)^ I Tt') ^ Cst^A^, E((H„+i-H„)^ I .F^, ) Cst(a,e) 



n 



Hence, by Chebyshev's and Doob's martingale inequalities, for all 5 > 0, 
1,1, I ^ * I r ^ <- V- ('"g")* ^ Cst (logZ„„(0))« 

and a similar inequality holds on the maximum of k ^ Z„p(0), so that, for 

sufficiently large Z„o(0), P(T2 < Tq A Ti J JSio) < 1/3. 

Let us now make use of notation 1^'-', and M^'' from Section [5^ (with 
= li^j), and let i7± := Y^^^^^ , := and := M^''^^ = - V^. 

Then the processes (C^^)n5;0 are martingales and, using (j47|) . for all n ^ no, 

E((ll^„^ - Wtf I ^„o) ^ E ( f %^^7^f^) ^ E i (52) 
so that, if T := {maxfc^„(, [VF"^ — W^J ^ S,i e {+, — }} then, for all 6 > 0, 
P(T^ I :Fn,) ^ ^ (^^^^2) - 1 + Z„,(-2) - l) < 3 
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for sufficiently large Z„g(0). 

Now, on T, for all n < T, choosing 5 = (log2)/3, and again for sufficiently 
large ^no(O), 

log Z„i±2) ^ log Z„„ (±2) + Ut- Ut, + 5 ^ 25 + log (±2) + - V^^ 

n -jj- Z„_i(±l) 
A:=no+l A;=Z„g{±l) 

^ 3(^ + log ("^^^ii^") +log(logZ„(±l)) < log(logZ„(±l)) ^ log(logZ„(0)), 

Viogz„,(,(±i)y 

where we use in the fourth inequality that, if n < T, then T„ ^ — e/2 and 
a" G (a - e/2,a + e/2) so that Z„(0) ^ Z„(±l) log Z„(±l) if Z„„ ^ Cst(a,e). 
This completes the proof P(ri < Tq A T2 | J^nJ ^ P(T^|J^„J < 1/3 for large 

The estimates ([5T]) (resp. ([52]) ) imply that the G (resp. F)-adapted martin- 
gales (^„)„^z„o(o) and (H„)„^^^jo) (resp. 14^^) are bounded in and hence 
converge a.s. 

Therefore, on {T = 00}, (i)-(ii) hold, {an)n'^o and {Rn)n^o converge a.s. 
Note that Lemma [A. 1 1 implies more precisely, for all v < 1/2, H„ — Hqo = o{n~'^), 
hence On — Ooo = o{Zn{0)^'^). Thus, on {T = 00}, 

n-l IT 

iogz„(±2)^[/„± = K^ = j; 



^^^Z,(±2) + Zfc(0) 
""^ I;x,,.=+n /. „ / 1 



-^^Z,(±l)logZ, 
«^log(logZ„(±l)) = log(logn), 



- -^^Zfc(±l)logZ,(±l) V Vlog^fc(±l^ 



which proves (iii). 

Acknowledgments. We would like to thank the referees for very helpful 
comments. 
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